On the Rate of Convergence for the 
Pseudospectral Optimal Control of Feedback 
Linearizable Systems * 

O . 

O ■ Wei Kang 

Department of Applied Mathematics 
o I Naval Postgraduate School 

; Monterey, CA 93943 

\q • wkang@nps . edu 

u 

O 1 Abstract 

'"pi | Over the last decade, pseudospectral (PS) computational methods for nonlinear 

constrained optimal control have been applied to many industrial-strength problems, 
notably the recent zero-propcllant-mancuvering of the International Space Station per- 
formed by NASA. In this paper, we prove a theorem on the rate of convergence for the 
optimal cost computed using PS methods. It is a first proved convergence rate in the 
■ literature of PS optimal control. In addition to the high-order convergence rate, two 

CO | theorems are proved for the existence and convergence of the approximate solutions. 

CO . This paper contains several essential differences from existing papers on PS optimal 

control as well as some other direct computational methods. The proofs do not use 
necessary conditions of optimal control. Furthermore, we do not make coercivity type 
of assumptions. As a result, the theory does not require the local uniqueness of op- 
timal solutions. In addition, a restrictive assumption on the cluster points of discrete 



o 

I solutions made in existing convergence theorems are removed. 



X 



1 Introduction 

Despite the fact that optimal control is one of the oldest problems in the history of control 
theory, practical tools of solving nonlinear optimal control problems are limited. Preferably, 
a feedback control law is derived from a solution to the famously difficult Hamilton-Jacobi- 
Bellman (HJB) equation. However, analytic solutions of this partial differential equation 
can rarely be found for systems with nonlinear dynamics. Numerical approximation of such 
solutions suffers from the well-known curse of dimensionality and it is still an open problem 
for systems with moderately high dimension. A practical alternative is to compute one 
optimal trajectory at a time so that the difficulty of solving HJB equations is circumvented. 
Then this open-loop optimal control can be combined with an inner-loop tracking controller; 
or it can be utilized as a core instrument in a real-time feedback control architecture such 
as a moving horizon feedback. A critical challenge in this approach is to develop reliable 
and efficient computational methods that generate the required optimal trajectories. In 
this paper, we focus on some fundamental issues of pseudospectral computational optimal 
control methods. 
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As a result of significant progress in large-scale computational algorithms and nonlinear 
programming, the so-called direct computational methods have become popular for solving 
nonlinear optimal control problems [TJ H7J , particularly in aerospace applications |16t [T8] . 
In simple terms, in a direct method, the continuous-time problem of optimal control is 
discretized, and the resulting discretized optimization problem is solved by nonlinear pro- 
gramming algorithms. Over the last decade, pseudospectral (PS) methods have emerged as 
a popular direct methods for optimal control. They have been applied to many industrial- 
strength problems, notably the recent attitude maneuvers of the International Space Station 
performed by NASA. By following an attitude trajectory developed using PS optimal con- 
trol, the International Space Station (ISS) was maneuvered 180 degrees on March 3, 2007, 
by using the gyroscopes equipped on the ISS without propellant consumption. This single 
maneuver have saved NASA about one million dollars' worth of fuel [12] . The Legendre PS 
optimal control method has already been developed into software named DIDO, a MAT- 
LAB based package commercially available |19j . In addition, the next generation of the 
OTIS software package [15] will have the Legendre PS method as a problem solving option. 

PS methods have been widely applied in scientific computation for models governed 
by partial differential equations. The method is well known for being very efficient in 
approximating solutions of differential equations. However, despite its success and sev- 
eral decades of development, the intersection between PS methods and nonlinear optimal 
control becomes an active research area only after the mid-1990's ([5j [6]). As yet, many 
fundamental theoretical issues are still widely open. For the last decade, active research has 
been carried out in the effort of developing a theoretical foundation for PS optimal control 
methods. Among the research focuses, there are three fundamental issues, namely the state 
and costate approximation, the existence and convergence of approximate solutions, and 
the convergence rate. The general importance of these issues is not limited to PS meth- 
ods. They are essential to other computational methods suchlike those based on Euler [10] 
and Runge-Kutta [9] discretization. Similar to other direct computational optimal control 
methods, PS method are based upon the Karush-Kuhn- Tucker (KKT) conditions rather 
than the Pontryagin's Minimum Principle (MPM). In [6] and [8], a covector mapping was 
derived between the costate from KKT condition and the costate from PMP. The covector 
mapping facilitates a verification and validation of the computed solution. For the problem 
of convergence, some theorems were published in [7J; and then the results were generalized 
in [13] to problems with non-smooth control. 

Among the three fundamental issues mentioned above, the most belated activity of 
research is on the rate of convergence. In fact, there have been no results published on 
the convergence rate for PS optimal control methods. Although some results on the issue 
of convergence were proved in [7J and [13] , a main drawback of these results is the strong 
assumption in which the derivatives of the discrete approximate solutions are required to 
converge uniformly. In this paper, we prove a rate of convergence for the approximate 
optimal cost computed using PS methods. Then, we prove theorems on existence and 
convergence without the restrictive assumption made in [7J and [13j . In addition to the 
high-order convergence rate addressed in Section [3l which is the first proved convergence 
rate in the literature of PS optimal control, this paper contains several essential differences 
from existing papers on PS optimal control as well as some other direct computational 
methods. First of all, the proof is not based on necessary conditions of optimal control. 
Furthermore, we do not make coercivity type of assumptions. As a result, the theory does 
not require the local uniqueness of optimal solutions. Therefore, it is applicable to problems 
with multiple optimal solutions. Secondly, the proof is not build on the bases of consistent 
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approximation theory [T7]. Thus, we can remove the assumption in [7] and [13] on the 
existence of cluster points for the derivatives of discrete solutions. The key that makes these 
differences possible is that we introduce a set of sophisticated regularization conditions in the 
discretization so that the computational algorithm has a greater control of the boundedness 
of the approximate solutions and their derivatives. Different from the existing results in the 
literature of direct methods for optimal control, the desired boundedness is achieved not by 
making assumptions on the original system, but by implementing specially designed search 
region for the discrete problem of nonlinear programming. This new boundary of search 
region automatically excludes possible bad solutions that are numerically unstable. 

The paper is organized as follows. In Section [21 the formulations of the optimal control 
problem and its PS discretization are introduced. In[3l we prove two theorems on the rate 
of convergence. In Section two theorems on the existence and convergence are proved. 

2 Problem Formulation 

For the rate of convergence, we focus on the following Bolza problem of control systems 
in the feedback linearizable normal form. A more complicated problem with constraints is 
studied in Section 0] for the existence and convergence of approximate solutions. 

Problem B: Determine the state-control function pair (x(t),u(t)), x G W and m 6 3?, 
that minimizes the cost function 

J(x(-),n(-)) = J 1 F(x(t),u(t)) dt + E(x(-l),x(l)) (2.1) 

subject to the following differential equations and initial condition 

±i = x 2 

< . : (2-2) 

X f — \ — JL f 

k x r = f(x) + g(x)u 
x(-l)=x (2.3) 

where x G 5ft r , u G K, and F : W x K -» K, E : W x W -» 5ft, / : W -» and g : W -» 5ft 
are all Lipschitz continuous functions with respect to their arguments. In addition, we 
assume g{x) / for all x. 

Throughout the paper we make extensive use of Sobolev spaces, W m ' p , that consists of 
functions, £ : [—1, 1] — > M whose j-th order weak derivative, lies in LP for all < j < m 
with the norm, 

m 

|| £ \\w m >p = || £^ \\lp 

3=0 

In this paper, we only consider the problems that have at least one optimal solution in which 
x*(t) has bounded m-th order weak derivative, i.e. x*(t) is in W™ 1 ' 00 . For some results, we 
assume m > 3. For others, m is smaller. Unless the term 'strong derivative' is emphasized, 
all derivatives in the paper are in the weak sense. 

The PS optimal control method addressed in this paper is an efficient direct method. In 
typical direct methods, the original optimal control problem, not the associated necessary 
conditions, is discretized to formulate a nonlinear programming problem. The accuracy of 
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the discretization is largely determined by the accuracy of the underlying approximation 
method. Given any function f(t) : [a,b] — > 9ft, a conventional method of approximation 
is to interpolate at uniformly spaced nodes: to = a, t± = (b — a)/N, ■ ■ ■ , ijy = b. How- 
ever, it is known that uniform spacing is not efficient. More sophisticated node selection 
methods are able to achieve significantly improved accuracy with fewer nodes. It is im- 
portant to emphasize that, for optimal control problems, the rate of convergence is not 
merely an issue of efficiency; more importantly it is about feasibility An increased num- 
ber of nodes in discretization results in a higher dimension in the nonlinear programming 
problem. A computational method becomes practically infeasible when the dimension and 
complexity of the nonlinear programming exceed the available computational power. In a 
PS approximation based on Legendre-Gauss-Lobatto (LGL) quadrature nodes, a function 
fit) is approximated by iV-th order Lagrange polynomials using the interpolation at these 
nodes. The LGL nodes, to = — 1 < ti < • ■ • < ijv = 1 5 are defined by 

to = — 1, ijv = 1) an d 

for k = 1,2, . . . , N — l,tk are the roots of L^(t) 

where ijjv(i) is the derivative of the iV-th order Legendre polynomial Ljv(t). The discretiza- 
tion works in the interval of [—1, 1]. An example of LGL nodes with N = 16 is shown in 
Figure [U It was proved in approximation theory that the polynomial interpolation at the 
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Figure 1: LGL nodes N = 16 

LGL nodes converges to f(t) under L 2 norm at the rate of 1/N m , where m is the smoothness 
of fit) (see for instance [3] Section 5.4). If fit) is C°°, then the polynomial interpolation 
at the LGL nodes converges at a spectral rate, i.e. it is faster than any given polynomial 
rate. This is a very impressive convergence rate. 

PS methods have been widely applied in scientific computation for models governed 
by partial differential equations, such as complex fluid dynamics. However, PS optimal 
control has several fundamental differences from the computation of PDEs. Solving optimal 
control problems asks for the approximation of several objects collectively, including the 
differential equation that defines the control system, the integration in the cost function, 
and the state and control trajectories. In addition to the various types of approximations, a 
nonlinear programming must be applied to the overall discretized optimization problem to 
find an approximate optimal control. All these factors may deteriorate the final approximate 
solution. The existing theory of PS approximation of differential equations is not applicable. 
New theory needs to be developed for the existence, convergence, and the rate of convergence 
for optimal control problems. 

In the following, we introduce the notations used in this paper. Then, the discretized 
nonlinear programming problem is formulated. In a PS optimal control method, the state 
and control functions, x(t) and it(t), are approximated by N-th order Lagrange polynomials 
based on the interpolation at the LGL quadrature nodes. In the discretization, the state 
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variables are approximated by the vectors x E 3ft r , i.e. 



Mk 



Mk 
J l 

Mk 



Mk 



is an approximation of x{tfS). Similarly, u Nk is the approximation of it(ifc). Thus, a discrete 
approximation of the function Xi(t) is the vector 



[ 



-Ml g m 



-MN 



A continuous approximation is defined by its polynomial interpolation, denoted by x^(t) 



i.e. 



N 



Xi(t) 



M 



(t) = JX fc &(t), 



(2.4) 



k=0 



where <f>k(t) is the Lagrange interpolating polynomial [I]. Instead of polynomial interpola- 
tion, the control input is approximated by the following non-polynomial interpolation 



u N (t) 



x? jt) -f{x" jt)) 

g(x N (t)) 



(2.5) 



In the notations, the discrete variables are denoted by letters with an upper bar, such as 



Mk 



and u . If k in the superscript and/or i in the subscript are missing, it represents the 
corresponding vector or matrix in which the indices run from minimum to maximum. For 
example, 



x 



-N 



Nk 



NO =AT1 



Nk 
1 

Nk 



-„Nk 



NO 
1 

NO 



siVO 



-„Nl 
-Ml 



-Ml 



MN 



NN 
1 

NN 



-MN 



Similarly, 



u 



N 



[ 



U 



NO 



-Ml 



U 



NN 



Given a discrete approximation of a continuous function, the interpolation is denoted by 
the same notation without the upper bar. For example, x^(t) in (|2,4p . u N (t) in (|2.5p . The 
superscript N represents the number of LGL nodes used in the approximation. Throughout 
this paper, the interpolation of (x ,u ) is defined by (|2.4p - (|2.5p . in which u N (t) is not 
necessarily a polynomial. It is proved in Lemma [5] that (12. 5h is indeed an interpolation. 
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Existing results in the analysis of spectral methods show that PS method is an approach 
that is easy and accurate in the approximation of smooth functions, integrations, and dif- 
ferentiations, all critical to optimal control problems. For differentiation, the derivative of 
x^(t) at the LGL node tk is easily computed by the following matrix multiplication [3] 



where the (N + 1) x (N + 1) differentiation matrix D is defined by 

( L N (tj) 1 -r • / 7. 

L N (t k ) U-t k ' 



(2.6) 



D 



tk 



N(N+1) 



N(N+1) 
4 ■ 



if i = k = 0; 
if i = k = N; 
otherwise 

The cost functional J[x(-),u(-)] is approximated by the Gauss-Lobatto integration rule, 



N 



J[x(-),u(-)] « J^,^) 



)w k + £(z 7 



fc=0 



where w k are the LGL weights defined by 



N(N + l) [L N {t k )f 



The approximation is so accurate that it has zero error if the integrand function is a polyno- 
mial of degree less than or equal to 2N — 1, a degree that is almost a double of the number 
of nodes [4j. Now, we are ready to define Problem B N , a PS discretization of Problem B. 

For any integer m\ > 0, let {a^ (mi) , (mi) , • • • ,aJJ_ r _ mi+1 (mi)} denote the coeffi- 
cients in the Legendre polynomial expansion for the interpolation polynomial of the vector 
x^f (D T ) mi . Note that the interpolation of x^ (D T ) mi equals the polynomial of - — ■ 
Thus, there are only N — r — m\ + 2 nonzero spectral coefficients because it is proved in 
Section [3] that the order of d is at most degree of N — r — m\ + 1. These coefficients 

depend linearly on x^f [3], 

a JV-r- mi + l( m l) 
2 

N — r — mi 4- 1 - 



L N -T--m. 1 +l( i 0) 



^oC*jv) 

L N -r-m 1 + l( t N) 



The PS discretization of Problem B N is defined as follows. 

Problem B N : Find x Nk € W and u Nk e 3ft, k = 0,1,..., N, that minimize 



(2.7) 



N 



J N (x N ,u N ) 



NO =AW\ 



Y,F(x Nk ,u Nk )w k + E(x N[i ,x 

k=0 



(2-f 



subject to 



f D(x?) T 
D(x?) T 



x N0 = x 



" f(x m ) + g(x m )u m 
f(x NN ) + g(x NN )u NN 



(2.9) 



(2.10) 



b < 



Nk 
Nk 



< b, for all < k < N 



(2.11) 



bj < [ 1 • • • ] D j (x^) T < bj, if 1 < j < mi - 1 and mi > 2 (2.12) 

N— r— mi+1 

E KMl^ ( 2 - 13 ) 

n=0 



Comparing to Problem B, (|2.9p is the discretization of the control system defined by the 
differential equation. The regularization condition (|2. 131) assures that the derivative of the 
interpolation up to the order of mi is bounded. It is proved in the following sections that 
the integer mi is closely related to the convergence rate. The inequalities (12. IIP . (|2.12p and 
(|2.13p are regularization conditions that do not exist in Problem B. It is proved in the next 
few sections that these additional constraints do not affect the feasibility of Problem B N . 
Therefore, it does not put an extra limit to the family of problems to be solved. 

In searching for a discrete optimal solution, it is standard for software packages of 
nonlinear programming to require a search region. Typically, the search region is defined 
by a constraint (|2. 1 1 j) . However, this box-shaped region may contain solutions that are not 
good approximations of the continuous-time solution. To guarantee the rate of convergence, 
the search region is refined to a smaller one by imposing constraints (|2.12p and (|2.13p . 
It is proved in this paper that there always exist feasible solutions that satisfy all the 
constraints and the optimal cost converges, provided the upper and lower bounds are large 
enough. In f|2. 12 j> . bj and bj represent the bounds of initial derivatives. In (|2.13p . d 

is the bound determined by satisfying the inequality (|3.ip . Without known the 

optimal solution, these bounds of search region have to be estimated before computation or 
they are determined by numerical experimentations. The constraints (|2.12p and (|2.13p are 
necessary to avoid the restrictive consistent approximation assumption made in [7]. At a 
more fundamental level, the order of derivatives, mi in (|2.13p . determines the convergence 
rate of the approximate optimal control. Another interesting fact that amply justify these 
additional constraints is that Problem B N may not even have an optimal solution if we do 
not enforce (|2.1ip . This is shown by the following counter example. 
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Example 1 Consider the following problem of optimal control. 

f 1 {x{t)-u{t)) 2 , 
mm / v , , A dt 

(x(.),«(o) 7-i u{ty 

x = u (2.14) 
x(-l) = e" 1 

It is easy to check that the optimal solution is 

u = e*, x(t) = e* 

and the optimal cost value is zero. Although the solution to the problem (|2.14p is simple and 
analytic, the PS discretization of (|2.14j) does not have an optimal solution if the constraint 
(|2.1ip is not enforced. To prove this claim, consider the PS discretization, 



D k (x N ) T ' 



" (~Nk _ n, (=N\T\* 

mm J (x ,u ) = 2^ — TTTTT^I w k 



k=0 

-N\T fr-.N\T 



D{x»Y ={u"f (2.15) 
x m = e' 1 

where D k is the fcth row of the differentiation matrix D. Let x N (t) be the interpolation 
polynomial of x N , then it is obvious that 

x N (t) - x N (t) ^ 

Thus, there exists k so that 

x Nk - D k (x N f ^ 

So, 

J N (x N ,u N )>0 (2.16) 
for all feasible pairs (x N , u ). For any a > 0, define 



X 



e 1 + a(t k + l) 



The interpolation of x N is the linear polynomial 

x N (t) = e" 1 + a(t + 1) 

Then, 

D k (x N ) T = x N (t k ) = a 

The cost function is 

J N (x N ,n N ) = V (e " 1 + afa 4 +1) " a)2 ^ 

a 4 

fc=o 

= E — -4 — 

fc=0 
fc=0 



2 



a 4 
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Therefore, J (x ,u ) can be arbitrarily small as a approaches oo. However, J (x ,u ) 
is always positive as shown by (|2.16p . We conclude that the discretization (|2,15p has no 
minimum value for J N (x N , u N ). 

3 Convergence Rate 

Given a solution to Problem B N , we use (|2.5[) to approximate the optimal control. In 
this section we prove that, under this approximate optimal control, the value of the cost 
function converges to the optimal cost of Problem B as the number of nodes is increased. 
More importantly, we can prove a high-order rate of convergence. In the literature, it has 
been proved that PS methods have a spectral rate when approximating C°° functions, i.e. 
the rate is faster than any polynomial rate. However, there are no results in the literature 
thus far on the convergence rate of PS optimal control. Meanwhile, in many problems solved 
by PS optimal control we clearly observed a rate of high-order in the convergence. In this 
section, we prove a convergence rate that depends on the smoothness of the optimal control. 
More specifically, the rate is about Ar2m 1 /3 _ 1 , where m is defined by the smoothness of the 
optimal trajectory. If the cost function can be accurately computed, then the convergence 
rate is improved to ^m-i ■ In the special case of C°°, it is proved that PS method is able to 
converge faster than any given polynomial rate. Before we introduce the main theorems of 
this section, the following example in [7J is briefly presented to show the rapid convergence 
of the PS optimal control method. 

Example 2 Consider the following nonlinear optimal control problem: 

Minimize J[x(-),u(-)] 

< Subject to x\(t) 

± 2 {t) 

{ On(o),x 2 (o)) 

It can be shown that the exact optimal control is defined by u*(t) = — ( 2 +f)3 ■ F° r this 
problem, the PS method achieves the accuracy in the magnitude of 10 -8 with only 18 
nodes. A detail comparison of the PS method with some other discretization methods are 
addressed in [7J. From Figure [2] in logarithmically scaled coordinates, it is obvious that the 
computation using the PS method converges exponentially. 

Problem B N has several bounds in its definition, b, b, bj, bj, and d. These bounds can 
be selected from a range determined by Problem B. The constraints b and b are lower and 
upper bounds so that the optimal trajectory of Problem B is contained in the interior of 
the region. Suppose Problem B has an optimal solution (x*(t), u*(t)) in which (x*(t))( m ^ 
has bounded variation for some m > 3, where x*{t) is the rth component of the optimal 
trajectory. Suppose m\ in Problem B N satisfies 2 < mi < m — 1. Then, we can select the 
bounds bj and bj so that (x*(t))v) is contained in the interior of the region. For d, we 
assume 

d > -^(U{x; imi+1) ) + V(x* r imi+1) ))C(3/2) (3.1) 



4xi(2) + x 2 (2)+4 f 
Jo 

u(t) 
(0,1) 



u 2 {t) dt 
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Figure 2: Error vs number of the nodes for the pseudospectral method 



where U(x* ( mi+1 )) is the upper bound and V{x* ( mi+1 )) is the total variation of x* (mi+1) (i); 
and £(s) is the C, function defined by 



oo 1 

cw=E^ (3-2) 



fe=i 

If all the bounds are selected as above, then it is proved in Section [5] that Problem B N is 
always feasible provided m > 2. Note that in practical computation, b, b, bj, bj, and d are 
unknown. They must be estimated based upon experience or other information about the 
system. 

Theorem 1 Suppose Problem B has an optimal solution (x*(t), u*(t)) in which the m-th 
order derivative (x*(t))( m ^ has a bounded variation for some m > 3. In Problem B N , select 
mi an d a so that 1 < m\ < m — 1 and < a < m\ — 1. Suppose /(•), <?(•), F(-), and 
E(xq, •) are C m and globally Lipschitz. Suppose all other bounds in Problem B N are large 
enough. Given any sequence 

{(x* N ,u* N )U> Nl (3.3) 

of optimal solutions of Problem B N . Then the approximate cost converge to the optimal 
value at the following rate 

\j(x*(.),u*(.))-J(x* N (.),u* N (.))\ < (]V _ r _ mi Ml 1)2m „ 2mi „ 1 +^ (3.4) 
\j( x *(.),u*(-))-J N (x* N ,u* N )\ < -, Ml -= 9 T + ^- (3.5) 

I V V ; J\ — (-jy _ r _ mi _ l)2m-2mi-l jya V > 

where M\ and M 2 are some constants independent of N . In ^3.4\ ), (x* N (t),u* N (t)) is the 
interpolation of jl3.3\) defined by \2.1$ - $2lty . In fact, x* N (t) is the trajectory of \2.S\) under 
the control input u* N (t). 

Theorem [1] implies that the costs of any sequence of discrete optimal solutions must 
converge to the optimal cost of Problem B, no matter the sequence of the discrete state 
and control trajectories converge or not. In other words, it is possible that the sequence of 
discrete optimal controls does not converge to a unique continuous-time control; meanwhile 
the costs using these approximate optimal controls converge to the true optimal cost of 
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Problem B. Therefore, this theorem does not require the local uniqueness of solutions for 
Problem B. This is different from many existing convergence theorems of computational 
optimal control, in which a unique optimal solution and coercivity are assumed. This is 
made possible because the proofs in this paper do not rely on the necessary conditions of 
optimal control. The key idea in the proof is to shape the search region in Problem B N by 
regulating the discrete solutions using (|2.1ip - (|2.12p - (|2.13p . We would like to emphasize that 
the regulation constraints are added to the discretized problem, not the original Problem 
B. So, the constraints do not restrict the problem to be solved, and they do not need to 
be verified before computation. In addition, increasing the number of constraints results in 
smaller search region for an optimal solution. 

Remark 3.1 If /(■), g(-), -^X") an d x *(t) are C°° , then we can select m and mi arbitrarily 
large. In this case, we can make the optimal cost of Problem B N converge faster than any 
given polynomial rate. 

Remark 3.2 From \3.4\ j and $3. 5\) . the convergence rate is determined by m, the smooth- 
ness of the optimal trajectory, and m\, the order in the regulation of discrete solutions. 
While m is a property of Problem B that cannot be changed, m\ in Problem B N can be 
selected within a range. However, the errors in \3.4\ j and \3. 5|) have two parts, one is an 
increasing function of mi and the other is a decreasing function of m\ . In Corollary we 
show an optimal selection of mi to maximize the combined convergence rate. 

The proof is convoluted involving results from several different areas, including nonlinear 
functional analysis, orthogonal polynomials, and approximation theory. First, we introduce 
the concept of Frechet derivative. Let us consider the continuous cost function, J(x(-),u(-)), 
subject to (|2.2p - (|2.3p as a nonlinear functional of u(-), denoted by J{u). For any u in the 
Banach space W m ~ 1,co , suppose there exists a linear bounded operator C: W m ~ 1,co — ► 3ft 



for all u + Au in an open neighborhood of u in W m ~ x '°°. Then, C is called the Frechet 
derivative of J{u) at it, denoted by J'{u) = C. If J'{u) exists at all points in an open subset 
of w m ~ 1,OQ , then J'{u) is a functional from this open set to the Banach space L(W m ~ l,oc , 5i) 
of all bounded linear operators. If this new functional has a Frechet derivative, then it is 
called the second order Frechet derivative, denoted by J"(u). The following lemma is 
standard in nonlinear functional analysis |21j . 

Lemma 1 Suppose J takes a local minimum value at u* . Suppose J has second order 
Frechet derivative at u* . Then, 



The rate of convergence for the spectral coefficients can be estimated by the following 
Jackson's Theorem. 

Lemma 2 (Jackson's Theorem }20^ ) Let h(t) be of bounded variation in [—1,1]. Define 



such that 



\J{u + Au) - J(u) - LAu\ = o(| |A«| \ W m-i, 



J(u* + Au) = {J"{u*)Au)Au + o(\\Au\ | 2 ) 
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then {a n }^ =0 , the sequence of spectral coefficients of H(t), satisfies the following inequality 

a n <-^=(U{h(t)) + V(h(t)))- 1 



jtt ' ' ' n 3 / 2 

for n > 1 . 

Given a continuous function h{t) defined on [—1,1]. Let p N (t) be the best polynomial 
of deg ree iV, i.e. the iVth order polynomial with the smallest distance to h(t^) under || • \ \oo 
norm. Let lNh(t) be the polynomial interpolation using the value of h(t) at the LGL nodes. 
Then, we have the following inequality from the theory of approximation and orthogonal 
polynomials [4], |llj . 



Lemma 3 

\\h(t) - iNhWoo < (1 + A N )\\h(t) -p N (t)\\oo 

where An is called Lebesgue constant. It satisfies 

A N < -log(N + 1) + 0.685 •• • 

7T 

The best polynomial approximation represents the closest polynomial to a function 
under || • W^. The error can be estimated by the following Lemma [4j. 

Lemma 4 (1) Suppose h(t) £ \Y m,0 ° . Let p N (t) be the best polynomial approximation. 
Then 



(t)-M*)lloo < 



c 



N m 

for some constant C independent of h(t), m and N . 
(2) Ifh(t) e W m ' 2 , then 

C\\h(t)\\ wm , a 



\\h(t) - Pxhit)^ < 



_/ym-3/4 



where Pat/i is the N-th order truncation of the Legendre series ofh(t). 

(3) If h(t) has the m-th order strong derivative with a bounded variation, then 

\W)-p N h(t)\\^< cv ^$) 

The following lemmas are proved specifically for PS optimal control methods. Similar 
results can be found in p3] except that some assumptions on m\ are relaxed. 

Lemma 5 (fl^j ) (i) For any trajectory, (x N ,u N ), of the dynamics h2.9\) . the pair (x (t), u N (t)) 
defined by \2.1$ - \2~1R) satisfies the differential equations defined in \2.2\) . Furthermore, 

x Nk = x N (t k ), u Nk = u N (t k ), for k = 0, 1, • • • , N (3.6) 

(ii) For any pair (x N (t), u N (t)) in which x N (t) consists of polynomials of degree less than 
or equal to N and u N (t) is a function, if (x N (t), u N (t)) satisfies the differential equations 
in pOp . then (x N ,u N ) defined by (Ej| satisfies WM . 

(Hi) If (x N ,u N ) satisfies \2.9\) . then the degree of xf {t) is less than or equal to N — i + 1. 
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Proof, (i) Suppose (x , u ) satisfies the equations in (|2.9p . Because x (t) is the polyno- 



mial interpolation of x , and because of equations (|2.6p . we have 



[if (^o) if(ii) 



if(*AT) 



xf D 1 

x i+l 
[ <1 



(to) 



Therefore, the polynomials xf(t) and xf_ x (i) must equal each other because they coincide 
at N + 1 points and because the degrees of xf (t) and xf +1 (t) are less than or equal to N. 
In addition, (|2.5|) . the definition of u N (t), implies the last equation in (|2.2j) . So, the pair 
(x N (t), u N (t)) satisfies all equations in (|2.2I) . Now, we prove (HEED. Because a^t) is an 
interpolation of x N , we know x"^ = x N (tk) for < k < N. From (|2.5p . 



Because of (j2.6|) . we have 

[xf(t ) 
Therefore, (|3,7|) is equivalent to 



xf [t N ) 



(3.7) 



D(xf) 



[ u N (t ) u N (h) ■■■ U N (t N ) ] 



T 



diag 



g(x 



NO) 



g[x 



NN S 



( 


- f(x N0 ) - 


D(xff - 




\ 


f(x NN ) 



Comparing to the last equation in (|2.9p . it is obvious that u N (tk) 
true. Part (i) is proved. 



-,Nk 



So, ID holds 



ii) Assume (x (t),u (t)) satisfies the differential equations in (|2.2p . Because x (t) are 



polynomials, (J2T6J) implies 



xfD T 



[if (to) if(ti) • 

[xf +1 (t ) Xf +1 (ti) 



if(iAr) 



(3.8) 



(tN) 



Furthermore, 



xfD T 



[xf(t ) xf{t x ) ••• xf(t N ) 
[ f(x N (t ))+g(x N (t ))u N (t ) 



(3.9) 



/(^(tiv)) + ff(x w (tiv))^(tiv 



Equations ()3.8j) and (|3.9p imply that (x ,u ) satisfies (|2.9p . Part (ii) is proved. 

(iii) We know that the degree of xf (i), the interpolation polynomial, is less than or 
equal to N. From (i), we know xf (t) = xf(t). Therefore, the degree of xf (i) must be less 
than or equal to N — 1. In general, the degree of xf (i) is less than or equal to iV — i + 1. □ 
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Lemma 6 Suppose {(x N , u N )}^ =Ni is a sequence satisfying h2.9\) . h2.11\) . h2.12\) and \2.13\) . 
where m\ > 1. Then, 

{||(x JV (t))W||oo|jV>JVi,i = 0,l,--- ,mi] 
is bounded. If f(x) and g(x) are C mi ~ l , then 

{||(«^(t)) (0 ||oo|^>JVi,Z = J l,-- - ,mi-l} 

is bounded. 

Proof. Consider (x^f (t))( mi \ From Lemma it is a polynomial of degree less than or 
equal to N — r — mi + 1. Therefore, 

JV-r-mi+l 

(a^(t)) (mi, = £ «n(™l)M*) 
n=0 

where L n (t) is the Legendre polynomial of degree n. It is known that |L n (i)| < 1. Therefore, 
(1XT3D implies that HOf (*)) (mi) ||oo is bounded by d for all N > N\. Now, let us consider 
Of (t))( mi_1 ). From (J2SD we have, 



(sc* (t))^ 1 " 1 ) = (a^ r (t))C'"i- 1 )| t== _ 1 + / (xf(s))( mi )ds 

Jo 

= [1 ••• ]D mi ~ 1 (x?) T + [ (x^(s))^ds 



So, \\(x^ (t))^ mi_1 ^|oo 5 N > Ni, is bounded because of (|2. 12j) . Similarly, we can prove all 
derivatives of x^f (t) of order less than mi are bounded. The same approach can also be 
applied to prove the bound 

N _ x?(t) - f(x N (t)) 
[t) ~ g(z»{t)) 

Because f(x) and g{x) have continuous derivatives of order less than or equal to m\ — 1, 
the boundedness of 

{ ll(^W) (0 lloo| N > N u j = 0,1,-- - ,mx - l} 

follows the boundedness of (x^f (t))^ proved above. □ 
Given any function hit) defined on [—1, 1]. In the following, U(h) represents an upper 
bound of h{t) and V{h) represents the total variation. 

Lemma 7 Let (x(t),u(t)) be a solution of the differential equation \2.2\) . Suppose Xr(t) 
has bounded variation for some m > 2. Let m\ be an integer satisfying 1 < m\ < m — 1. 
Then, there exist constants M > and N\ > so that for each integer N > Ni the differ- 
ential equation \2. 2\) has a solution (x N (t), u N (t)) in which x N (t) consists of polynomials 
of degree less than or equal to N. Furthermore, the pair (x (t),u (t)) satisfies 

ItfW—WIL < (N _ r !T^)Z-^ * = !•».•■• ,r (3.10) 

m^(t)f>-(, M f>^ < _ -Jfl^kgL^ , i = 1 , 2 ,.., mi( 3.ii) 

II ^ M\\x r \\ Wm ,2 

l|U W-«C*>lloo < (jv _ r _ mi + 1)(m - mi) - 3/4 ( 3 - 12 ) 
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Furthermore, the spectral coefficients of (x^)^ mi \t) satisfy 

tfWI <- ^ + , , , (3 , 3) 



7rn 

If f(x) andg(x) have Lipschitz continuous Lth order partial derivatives for some L < mi— 1, 

||(«*(t)) (,) -M*)) (0 lloo < ^IMI^ r ^ 7J , 1 = 1,-, L (3.14) 

Furi/iermore, 

x^(-l)=x(-l) f . 

u*(-l)=u(-l), If mi >2 1 j 

Remark 3.3 in £/us lemma, if x r (t) has the m-th order strong derivative and ifxr(t) 
has bounded variation for some m > 2, then the inequalities 113. 10\) . 113. 11\) . and \3.12i) are 
slightly tighter. 

H*f(*)-^)IU < {N _ r M "^- mi) -^ i = 1,2,--- ,r (3.16) 
ll(^(*))«-M*))«IU < (iy _ r jfeUgg^p^ , i = l,2,--- ,mi(3.17) 

11^(^-^)11^ < M||Xr|lw 7' 2 _ (3.18) 

II W - (AT - r - mi + l)(m-mi)-l/2 v ; 

The proof is identical as that of Lemma^ except that the error estimation in (3) of Lemma 
is used. 

Proof. Consider the Legendre series 

N—r—mi+l 

(x r )^\t)~ a n(mi)L n (t) 

n=0 

A sequence of polynomials x± (t), ■ ■ ■ , x^ +m (t) is denned as follows, 

Xr+miit) = Yl a%(mi)L n {t) 

n=0 

^ +mi _i(t) = (x r )^~ 1 \-l)+ f x? +mi {s)ds 



1 



x r+li t ) = %r{— 1) + / X^ +2 {s)ds 



and 



X j 



(-1) + 



N +1 (s)ds, 



for 1 < i < 
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Define 

x N (t)=[x?(t) ■■■ x?(t)f 



and define 



u (t) = - 



g(x N (t)) 

From the definition ofx N (t), we have x N ( — 1) = x{— 1). If mi > 2, then x r +i(— 1) = x r {— 1). 
From the definition of u N (t), we know u N (—l) = u(— 1) provided mi > 2. Therefore, 
(x N (t),u N (t)) satisfies (^T5l) . It is obvious that xf (t) is a polynomial of degree less than 
or equal to iV; and (x N (t), u N (t)) satisfies the differential equation (|2.2p . Because we assume 
< oo, we have i'™^ 6 L 2 . From Lemma [5] 

AT— r-mi+l 

||x r VW-4 mi) Wlloo = \\x^(t)- Yl «n(^l)in(t)||oo 

n=0 

< C r i||x r || W r m ,a(iV - r - mi + i)-("i)+3/4 
for some constant C\ > 0. Therefore, 

l^-iW-CXr)^ 1 - 1 )^)! < f \x? +mi ( S ) - { Xr )^\s)\ds 

< 2Ci||x r || W rm, 2 (JV - r - mi + i)-("™)+3/ 4 

Similarly, we can prove (|3.10p and (13. lip . 

To prove (|3.13p . note that the spectral coefficient (mi) of (x^)( mi > (t) is the same as 
the spectral coefficients of (x r )^ mi ' ) (t). From Jackson's Theorem (Lemma [2]), we have 

|a^(mi)| < A( [/(x (- 1+ i)) + y(4- 1+ i) )) 1 
V7T n J/ 



In a bounded set around x(t), we have g(x) > a > for some a > because / and g 
are Lipschitz continuous (Definition of Problem B). Therefore, the function 

s ~ f{x) 



9{x) 

is Lipschitz in a neighborhood of (x,s), i.e. there exists a constant C2 independent of N 
such that 



u N {t)-u{t)\ 



x? +1 (t)-f(x»(t)) x r {t)-f{x{t)) 



g(x"(t)) g(x(t)) 
< C 2 (\x? +1 (t) - x r (t)\ + \x?(t) - Xl (t)\ + ■■■ + \x?(t) - x r (i)|X3-19) 



Hence, (I3TT2"|1 follows ([3TTUD . (IXTTD and (I3TTU|) when / = 0. Similarly, we can prove (I3TTD 
for I < L. □ 
Now, only after this lengthy work of preparation, we are ready to prove Theorem [TJ 
Proof of Theorem^ Let (x* (t) , u* (t)) be an optimal solution to Problem B. According 
to Lemma [7] and Remark 13.31 for any positive integer N that is large enough, there exists a 
pair of functions (x N (t), u N (t)) in which x N (t) consists of polynomials of degree less than or 
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equal to N. Furthermore, the pair satisfies the differential equation with initial conditions 
in Problem B and 



H*"W-^WII- < {N _ T _ mi L +1) ™-* < 3 - 21 > 

||(^(())C -W(«)) ( "IU < (JV _ r _, ni L +1) „,-„,- 1/2 . 1 <'<■»! (3-22) 
If we define 

U Nk =u N {t k )rx Nk = x N {t k ) 

Then {(x N ,u N )} satisfies (gSJ) and dZTD|> (Lemma O and [7} . Because s^(t) is a polynomial 
of degree less than or equal to N and because of (|2.6p . we know (sc^ (t))^ equals the 
interpolation polynomial of x^ (D T y . So, 

[i o ••• o]jy(^) T = (^(t))W| t= _i 

Therefore, (|3.22p implies (|2. 12j) if the bounds 6^ and are large enough. In addition, the 
spectral coefficients of x^ (D T ) mi is the same as the spectral coefficients of (x^ (t))^ mi \ 
From ipngjl . ([OD and ffSTT]> . we have 

iV— r— 

£ \ a n(mi)\<d 
n=0 

So, the spectral coefficients of (x^)( mi ^ satisfies ()2.13p . Because we select b and b large 
enough so that the optimal trajectory of the original continuous-time problem is contained 
in the interior of the region, then (|3.20p and (13.211) imply (|2.1ip for N large enough. In sum- 
mary, we have proved that (x N ,u N ) is a discrete feasible trajectory satisfying all constraints, 
HZ2D-H2ZI3D, in Problem B N . 

Given any bounded control input u(-), because the system is globally Lipschitz, it 
uniquely determines the trajectory x(-) if the initial state is fixed. Therefore, the cost 
J(x*(-),u*(-)) can be considered as a functional, denoted by J{u). Because all the func- 
tions in Problem B are C m with m > 2, we know that J(u) has second order Frechet 
derivative. By Lemma [1] 

\J(x*(-),u*(-))-J(x N (-),u N (-))\ 
= \J(u*)-J(u N )\ 
< C^Wu* -u N \\ 2 wmi -^) 



C2_ 

(N- r - mi + l) 2m -2mi- 



- /Ar I — , i\2m-2m,-l (3.23) 



for some constant numbers C\ and Ci. The last inequality is from (|3.2ip . 

Now, consider F(x N (t),u N (t)) as a function of t. Let F N (t) represent the polynomial 
interpolation of this function at t = to, t\, ■ ■ ■ , i/v- Let p(t) be the best polynomial approx- 
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imation of F(x (t),ii (t)) under the norm of L°°[—l, 1]. Then we have 



\J(x»(.),u»(.))-J N (Z N ,ii N - 

N 

\J(x N (.),u N (.)) -Y,F(x Nk ^ Nk )w k - E(x m ,x 



NO 



k=0 



F(x N (t),u N (t))dt- I F N (t)dt 



?N i 



-1 



< 



F(x N (t),u N (t))- F N (t)\dt 



< 2(l + A N )\\p(t)-F(x N (t),u N (t))\\ 



where 



Atv < -logiN + 1) + 0.685' 

7T 



(3.24) 



(3.25) 



is the Lebesgue constant. The inequality (13.241) is a corollary of Lemma [3l Because /(•), 
g(-), and F(-) are C m , it is known (Lemma H|) that the best polynomial approximation 
satisfies 



[t)-F(x N (t),u N (t 



< 



C 3 



f^\\F(x N (t),u N (t))\\ wmi - l!C 



N r 



Because of Lemma El {\\F(x N (t),u N (t))\\ jymj-i.oc | N > A^i} is bounded. Therefore, 



\j(x N (-),u N (-)) - J N (£ N ,<a N )\ < (1 + Ajv)C4 < ° 5 



jymi-l — jy c 

for some constant numbers C4 and C5 independent of N and any a < mi — 1. Let 



{(x* N ,u* N )}°° 



N=N 



(3.26) 



(3.27) 



be a sequence of optimal discrete solutions. Its interpolation is denoted by (x* N (t), u* N (t)). 
Then, similar to the derivation above, we can prove 



\J(x* N (-),u* N (-)) - J N (x* N ,u* N )\ 

< 2(l + A N )\\p N (t)-F(x* N (t),u* N (t))\U 
C&(1 + A N ) N N 



(3.28) 



< 



\F(x*»(t),u*»(t))\\ 



where p N (t) is the best polynomial approximation of F(x* N (t),u* N (t)) with degree less than 
or equal to N. Because of Lemma[6l | \F(x* N (t), u* N (t))\ \ w „ n -i,oo \N > Ni} is bounded. So 



\J(x* N (.),u* N (.))- J N (x* N ,u* N )\ <^ 



(3.29) 
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for some constant C-? > 0. Now, we are ready to piece together the puzzle of inequalities 
and finalize the proof. 



J(x*(0, «*(•)) 

< J(x* N {-),u* N (-)) 

< J N (x 



< J N (x i \u N ) + 



Cj_ 

N a 



C5 , C7 



< J(s"(.),fi"(-))4 

< J(x*(-), «*(■)) + 



C 2 



C5 _cv_ 



(JV 



2j2m-2mi-l 



Therefore, 

0< J(x* N (-),u* N (-))- J(x*(.), «*(•))< 



(AT 



(x^^u*^)) is a feasible 
trajectory (Lemma [5|) 

( inequality ([3^9]) ) 

/ (x N ,u N ) is a feasible discrete 

I trajectory and {x* ,u* ) is optimal 

( inequality p6| ) 



( inequality (IT23j) ) 



C2 _Cs_ C7 

mi _ ]\2m-2mi-l jyo J\[a 



This inequality implies (|3.4p . Furthermore, (|3.4p and (|3,29p imply (|3.5p , □ 
According to Theorem [TJ the convergence rate of the approximate cost is determined by 
two terms with the rates 

1 1 



(N 



m ^ ]^2m— 2m\ — 1 J^2m— 2m\— 1 



and 



1 



1 



(3.30) 
(3.31) 



where m, the smoothness of x*(t), is fixed. However, mi can be selected provided /(•), 
g(-), and T(-) are smooth enough. Note that increasing mi will increase the rate defined 
by (|3.3ip . but decrease the rate defined by (|3.30p . There is a value of mi that determines 
the maximum rate. Given any real number a £ 3ft, let [a] be the greatest integer less than 
or equal to a. 

Corollary 1 Under the same assumption as Theorem^ the convergence rate of J(x* N (-),u* N i 
and J N (x* N ,u* N ) is 

1 



O 



in which 



1 < 7 < § 

3(1-7) 7>f 
1 — any positive number, 7 = 



where 7 = — [^p]. To achieve this rate, 







2m 






3 


mi = < 










2m 






3 



o< 7 <4 



+ 1, 7>4 
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Proof. The optimal convergence rate is determined by 

max min{2m — 2m\ — 1, mi 

2<m\ <m—l 

The maxmin is achieved at 

mi 



1} 



2m 
T 



However, it may not be an integer. If mi is not an integer, we have two options, 



mi 



If we define 



2m 
3 



7 or mi 



2m 
3 



then either mi 

min{2m — 2m% — 1, mi — 1} 



,2m, ,2m, 
[-]or[-] + l 



2m r 2m 

7 + 1. It is straightforward to verify that 

2f- 7 -l, ™i = f -7 

2S -7-3(1-7), mi = 2f-7 + l 



Therefore, — 1 — 7 is larger when 7 < |, and ^p — 1 — 2(1 — 7) is larger if 7 > |. The 
special case at 7 = is because of (|3.3ip when ^ equals an integer. □ 
Different from numerical computations of differential equations, solving an optimal con- 
trol problem requires the approximation, (|2.1[) . of the integration as an addition to the 
approximation, (|2.9p . of the differential equation. The contributions of these approxima- 
tions to the overall approximation error are different; and the errors are inversely related to 
each other. The following theorem indicates that the rate (|3.30p is due to the approximation 
error of the differential equation and the rate (I3,31j) is due to the approximation error of the 
quadrature integration rule (|4.5p . To verify this fact, we define the following discretization 
problem with exact integration. 



Problem B N (J) Find x Nk G K r and u Nk € 3?, k 

J(x N (-),u N (-)) 



0, 1, . . . , N, that minimize 
F(x N (t),u N (t)) dt + E{x N (-l),x N (l)) 



(3.32) 



subject to 



D(x?) T 
D(x») T 

D(x?_i) T 

D(Xr) T 



x m = xo 



(x r ) 



f(x N0 ) + g(x m )u 



N0\„-,N0 



(3.33) 



f(x NN )+ g{x NN )u 



NN\ n -,NN 



(3.34) 



b < 



-,Nk 



< b, for all < k < N 



bj < [ 1 • • • ] D j {x?) T < bj, 1 < j < mi - 1 

N— r— mi+1 

E \ a n(mi)\<d 



n=0 



(3.35) 
(3.36) 

(3.37) 



20 



In Problem B (J), (x (t), u (t)) is the interpolation of (x , u ). In this discretization, 
we approximate the differential equation by the PS method. However, the integration in the 
cost function is exact. In this case, the overall error is controlled by the single rate (|3.30p 
rather than the two-rate convergence of Problem B N . Without the integration error of the 

cost function, the convergence rate is improved to ^ 2 m-3 ' anc ^ ^ e smo °thness requirement 
can be reduced to m > 2. 

Theorem 2 Suppose Problem B has an optimal solution (x* (t) , u* (t)) in which the strong 
derivative (x*(t))( m ^ has bounded variation for some m > 2. In Problem B N (J), select mi 
so that 1 < mi < m — 1. Suppose /(•), g(-), and F(-) are C m . Suppose all other bounds in 
Problem B N are large enough. Given any sequence 

{(x* N ,u* N )} N > Nl (3.38) 

of optimal solutions of Problem B (J). Then the cost of h3. 38\) converges to the optimal 
cost at the following rate 

\j(x*(.),u*(.))-J(x* N (.),u* N (.))\ < (N _ r _ m M \ )2m _ 2mi _i (3-39) 



for some constants Mi independent of N. 
Proof. Let 

{(x* N ,u* N )}% =No (3.40) 

be a sequence of optimal solutions of Problem B N (J). According to Lemma [7] and Re- 
mark 13.31 for any positive integer N that is large enough, there exists a pair of functions 
(x N (t), u N (t)) in which x N (t) consists of polynomials of degree less than or equal to N. 
Furthermore, the pair satisfies the differential equation in Problem B and the inequalities 
(13301) . (I3T2TT) . and (13321) . If we define 

ti Nk = u N (t k ),x. Nk = x N (t k ) 

Then, from the first part in the proof of Theorem [H (x N ,u N ) is a discrete feasible solution 
satisfying all constraints in Problem B N (J). By Lemma Q] 

\J(x*(-),u*(-))-J(x N (-),u N (-))\ 
= \J(u*)-J(u N )\ 
< Ci(||u* -u N \\ 2 wmi - h oo) 

C2 

(N -r - mi + l)2m-2roi- 

for some constant numbers C\ and C 2 . The last inequality is from (|3.12p . The interpolation 
(x* N (t), u* N (t)) of (|3.40p is a feasible trajectory of Problem B (Lemma|5|. Thus, 

J(x*(0, «*(■)) 

(x* N (t),u* N (t)) is a feasible 



< ,»t - - ^ m -2m^l ( 3 - 41 ) 



< J(x* N (-),u* N (-)) 



trajectory (Lemma [5]) 



(x N ,u N ) is a feasible discrete 
< J(a; iV (-),n JV (-)) L^t^'^Af^N 



C2 

(N -r -m x - If 



trajectory and (x* , u* ) is optimal 



< J( x *(.), u *(.)) + — - - 1 m _ 2mi _ 1 ( inequality (HUD) 
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Therefore, 

0< J(x* n (-),u* n (-)) 



*N r \ „.*N f \ C2 



(N -r -mi - l)2m-2mi-l 

□ 



4 Existence and Convergence of Approximate Optimal Solu- 
tions 

In Section [3l the rate of convergence for the cost function is proved. However, the results do 
not guarantee the convergence of the approximate optimal trajectory {(x (i), u N (t))}. In 
this section, we prove the existence of feasible trajectories for Problem B N and the existence 
of a convergent subsequence in any set of approximate optimal solutions. In addition, we 
consider a larger family of problems. Different from Section [2] where Problem B does not 
contain constraints other than the control system, in this section the problem of optimal 
control may contain nonlinear path constraints. Furthermore, general endpoint conditions 
are allowed, rather than being limited to the initial value problem as in the previous sections. 

Problem B: Determine the state-control function pair (x(t),u(t)), x € R r and m £ 3?, 
that minimizes the cost function 

J(x(-),«(0) = J l J(x(t),u(t))dt + E(x(-l),x(l)) (4.1) 

subject to the state equation 

±1 = X2 

. _ (4-2) 
x r = f(x) +g{x)u 
end-point conditions 

e(x(-l),x(l)) = (4.3) 

and state-control constraints 

h(x{t),u(t)) < (4.4) 

where x € u € R, and F : W x K -» R, E : W x W -» R, / : W -» R, 5 : K r -» K 
e : R r x R r — > R^* 2 and /i : R r x R — > R^ are all Lipschitz continuous functions with 
respect to their arguments. In addition, we assume g{x) ^ for all x. The corresponding 
discretization is defined as follows. 

Problem B N : Find x Nk G R r and u Nk £ R, fc = 0, 1, . . . , N, that minimize 

iv 

j"(x N ,u N ) = ^F(x Nk ,u Nk )w k + E(x m ,x NN ) (4.5) 

fc=0 
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subject to 



( D(x^) T 
D{x») T 

D(x?) T 



<~N\T 



N\T 



f(x m ) + g(x N ")u 



N0\ n -,N0 



NN\„-,NN 



f(x"")+g(x" N )u 



||e(x Ar0 ,x^)|| oo <(iV-r-l)-' 3 
h(x Nk ,u Nk ) < (N - r - l)' 13 ■ 1, 



for all < k < N 



(4.6) 



(4.7) 
(4.8) 



b < 



x Nk 
u Nk 



< b, for all < k < N (4.9) 



bj < [ 1 • • • ] D j {x 1 ; ) T < bj, if 1 < j < mi - 1 and mi > 2 (4.10) 

N-r-m-i+l 
n=0 

The discretization is almost identical to the one used in the previous sections except for 
the path constraints and the endpoint conditions, which must be treated with care. Note 
that in Problem B N the right sides of (|4.7|) and (|4.8|) are not zero. It is necessary to relax 
(|4.3j) and (|4,4p by a small margin for the reason of feasibility. The margin approaches zero 
as N is increased. Without this relaxation, it is shown by a counter example in [7] that 
Problem B N may have no feasible trajectories. 

Some feasibility and convergence results were proved in [7j, which take the form of 
consistent approximation theory based on the convergence assumption about {x^ (t)} and 
{x N0 }. The goal of this section is to remove this bothersome assumption by using a fun- 
damentally different approach. In addition, the proofs in this section are not based on 
necessary conditions of optimal control and any coercivity assumption, which are widely 
used in existing work on the convergence of direct optimal control methods. Before we 
introduce main results in this section, some useful results from [7] are summarized in the 
following Lemma. 

Lemma 8 (^jj) Suppose Problem B has an optimal solution (x* (t) , u* (t)) satisfying x*(t) G 
W™' 00 , m > 2. Let {(x N , u N )}^ =N be a sequence of feasible solutions to {4.6[ )- $~lfy . Sup- 
pose there is a subsequence {Nj}J^ =1 of {N}^ =1 such that the sequence converges 

as Nj — > oo. Suppose there exists a continuous function q(t) such that x r j (t) converges to 
q(t) uniformly in [—1,1]. Then, there exists (x°° (t) , u°° (t)) satisfying |J~2|)-|73P such that 
the following limits converge uniformly in [—1, 1]. 

lim (x N i(t) -x°°(t)) = (4.12) 



lim (u N i(t)-u°°(t)) = (4.13) 
lim J N *(x N i,u N *) = J(x(-),u(-)) (4.14) 
lim J(x N \u N J) = J(x(-),u(-)) (4.15) 
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In addition to the above assumptions, if {(x N ,u N )}^ =Nl is a sequence of optimal solutions 
subject to the constraints fl^.ffi )- j^79[ ), then (x°°(t), u°°(t)) must be an optimal solution to 
Problem B. 

The following are the two main theorems of this section. Relative to [H], these results 
has a tightened bounds for mi and (5. 

Theorem 3 (Existence of solutions) Consider Problem B and Problem B N defined in 
Section^ Suppose Problem B has a feasible trajectory (x(t),u(t)) in which (x r (£))( m ) has 
bounded variation for some m > 2. In Problem B N , let m\ be any integer and (3 be any 
real number satisfying 1 < mi < m — 1 and < j3 < (m — mi) — |. Then, there exists 
Ni > so that, for all N > N\, Problem B N has a feasible trajectory satisfying M-fty - ^.H ). 
Furthermore, (x(t),u(t)) and the interpolation (x N (t), u N (t)) satisfy \3.10\) - [3.13\) . 

Theorem 4 (Convergence) Consider Problem B and Problem B N defined in Section^ 
Suppose Problem B has an optimal solution (x*(t), u*(t)) in which (x*(t))^ has bounded 
variation for some m > 3. In Problem B N , let mi be any integer and j3 be any real 
number satisfying 2 < mi < m — 1 and < j3 < (m — mi) — |. Then for any se- 
quence {(x* N ,u* N )}^ =N of optimal solutions of Problem B N , there exists a subsequence, 

m 



{(x* N: > ', u* Nj )}p >1 , and an optimal solution, (x*(t), u*(t)), of Problem B so that the following 
limits converge uniformly in [—1, 1] 

lim (x* N '(t)-x*(t)) = 

Nj—>oo 

lim (v* N '(t)-v*(t)) = (4.16) 

Nj— >oo 

lim J N i(x* N i,u* N i) = J(x*(-), «*(•)) 
lim J(x* N >(.),u* N >(-)) = J(x*(-),u*(-)) 

where (x* N i (t),u* N i (t)) is the interpolation of (x* N ,u* N ). 

Remark 4.1 The integers m and mi in Theorem^ are smaller than those in Theorem 
^ i.e. the existence theorem is proved under a weaker smoothness assumption than the 
convergence theorem. 

To prove these theorems, we first briefly review some results on real analysis and then 
prove a lemma. Given a sequence of functions {/fc(i)}fcLi defined on [a, b]. It is said to be 
uniformly equicontinuous if for every e > 0, there exists a 5 > such that for all t, t' in 
[a, b] with \t' — t\ < 5, we have 

\fk(t) - f k (t')\ < e 

for all k > 1. The following Proposition and Theorem are standard in real analysis |20j . 

Proposition 1 If fk(t) is differentiable for all k, and if {/fe(i)}fcLi ^ s bounded. Then, 
{/fe(i)}2Ll * s uniformly equicontinuous. 

Theorem 5 (Arzela-Ascoli Theorem) Consider a sequence of continuous functions {/i n (i)}^Li 
defined on a closed interval [a, b] of the real line with real values. If this sequence is uni- 
formly bounded and uniformly equicontinuous, then it admits a subsequence which converges 
uniformly. 
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Lemma 9 ([T^D Let {{x N ,u N )}^ =Ni be a sequence satisfying |^,6p -( [7^ - Assume the set 

{l|2^(*)lloo|JV> JVi} (4-17) 



is bounded. Then, there exists (x°° (t) , u°° (t)) satisfying ^.Sfy - fc^j) and a subsequence 
{{x N ^u N ^)}^ } y Ni such that WW) , U^i) , (tW and (JJSj) hold. Furthermore, if {(x N ,u N )}^ =Nl 
is a sequence of optimal solutions to Problem B N , then (x°° (t) , u°° (t)) must be an optimal 
solution to Problem B. 

Proof. Let x^ (t) be the interpolation polynomial of x^ . Because (|4.17p is a bounded 
set, we know that the sequence of functions {x^ (t)\N > iVi} is uniformly equicontinuous 
(Proposition!!]). By the Arzela-Ascoli Theorem, a subsequence {xfi (t)} converges uniformly 
to a continuous function q(t). In addition, because of (|4.9|) . we can select the subsequence 
so that {x Nj0 }'N >N 1 is convergent. Therefore, all conclusions in Lemma [8] hold true. □ 

Now, we are ready to prove the theorems. 

Proof of Theorem^- For the feasible trajectory (x(t),u(t)), consider the pair (x N (t), u N (t)) 
in Lemma [7] that satisfies the differential equation (14. 2h . Define 



x Nk = x N (t k ) 
u Nk = u N {t k ) 



Nk _ „,N(+ \ (4-18) 



for < k < N. From Lemma El we know that {(x N , u N )} satisfies the discrete equations 
in (|4,6p . In the next we prove that the mixed state-control constraint (|4.8p is satisfied. 
Because h is Lipschitz continuous and because of (13.10|) and (|3.12p . there exists a constant 
C independent of N so that 

\\h(x(t),u(t)) - h(x N (t),U N (t))\\ < C{\ Xl {t) - X?(t)\ + ■■■ + \x r (t) - X?(t)\ + \u(t) - U N (t)\) 

< CMV\\x r \\ wm ,2(r + 1)(N -r-m 1 + i)-(™-™i)+3/4 

Hence 

h(x N (t),u N (t)) < h(x(t),u(t)) + CM I \x r \\ wm ,2 (r + l)(N-r-m 1 + i)-("i)+3/4 . 1 
< CM\\x r \\ Wm .2(r + 1)(N -r -mi + i)~(™~™0+3/4 

Because [i < m — mi — |, there exists a positive integer N\ such that, for all N > N±, 

CM\\x r \\ wm ,2(r + l)(N-r- mi + l)- im - mi)+3/4 < (N - r - 1)^ 

Therefore x± . . ., x^ u N (tk), k = 0, 1, . . . , N, satisfy the mixed state and control 
constraint P~8j) for all N > N 1 . 

By a similar procedure, we can prove that the endpoint condition (|4.7p is satisfied. 
Because x^ (t) is a polynomial of degree less than or equal to N, and because of (|2.6p and 
(|4.18p . we know (x^f (t))^ equals the interpolation polynomial of x^f {D T y . So, 

[1 ••• 0]Di(x?f = (*?(*)) W l*=-i 

Therefore, (|3.1ip implies (|4.10p if the interval between bj and bj is large enough. In addition, 
the spectral coefficients of x^f (D T ) mi is the same as the spectral coefficients of (x^f (t))^ mi \ 
From (|3.13p and (|3.1|) . we have 

JV— r-mi+l 

n=0 
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So, {(x N ,u N )} satisfies (|4.1ip . Because we select b and b large enough so that the optimal 
trajectory of the original continuous-time problem is contained in the interior of the region, 
we can assume that (x(t),u(t)) is also bounded by b and 5. Then, (|3.10p and (|3.12p imply 
(|4.9p for N large enough. To summarize, (x N ,u N ) satisfies (|4.6p - (|4.1ip . Therefore, it is a 
feasible trajectory of (|P JI -l|P j) . □ 

Proof of Theorem^ Consider {(x* N , u* N )}^ =Ni , a sequence of optimal solutions of 
Problem B N . From Lemma [6l 

{||^(t)||ao|JV>JVl} 

is bounded. Now, we can apply Lemma [9] to conclude that there exists a subsequence of 
{(x* N (t), u* N (t))}ft =Ni and an optimal solution of Problem B so that the limits in (|4.16p 
converge uniformly. □ 



5 Simulation results 

The rate of convergence for the optimal cost is illustrated in the following example 

rn 

min / (1 — x\ + x\X2 + X\u) 2 dt 

u Jo 
subject to 

x\ = — x\xi 

1 

X2 = — 1 H h X2 + sin t + u 

Xl 



' 1 " 




- l - 


, x(tt) = 


7T + 1 







2 



The analytic solution of this problem is known so that the approximation error can be 
computed 

xi(t) = z 

1 — sin t + t 

X2(t) = 1 — cost 

u = — (t + 1) + s'mt + cost 

optimal cost = 



The problem is solved using PS optimal control method. The approximated optimal cost 
is compared to the true value. The number of nodes, N, ranges from 4 to 16. The error 
decreases rapidly as shown in Table The rate of convergence is illustrated in the following 



N 


4 


6 


8 


10 


12 


14 


16 


Error 


7.5 x 1(T 2 


1.1 x 1(T 3 


2.1 x 1(T 4 


7.1 x 10~ 5 


6.7 x l(T e 


1.0 x 10-* 


5.8 x lO" 7 



Table 1: The error of optimal cost 

Figure El Because the analytic solution is C°°, the rate of convergence of the PS method 
is faster than any polynomial rate. As a result, it converges exponentially. Of course, in 
practical computations the accuracy is limited by the machine precision of the computers. 
Therefore, the accuracy cannot be improved after N is sufficiently large. 
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Figure 3: Log-scale plot of the error of optimal cost (the solid curve) 

6 Conclusions 

It is proved that the PS optimal control has a high-order rate of convergence. Accord- 
ing to the theorems in Section [31 the approximate cost computed using the Legendre PS 
method converges at an order determined by the smoothness of the original problem. More 
specifically, the rate is about Af2m 1 /3 _ 1 , where m is defined by the smoothness of the optimal 
trajectory. If the cost function can be accurately computed, then the convergence rate is 
improved to N -2m-i ■ If the optimal control is C°°, then the convergence rate can be made 
faster than any given polynomial rate. The results in Section [J] imply that the discretization 
using the Legendre PS method is feasible; and there always exists a convergent subsequence 
from the approximate discrete optimal solutions, provided some smoothness assumptions 
are satisfied. 
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